{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "\n",
    "# 线性回归的简洁实现\n",
    "\n",
    "随着深度学习框架的发展，开发深度学习应用变得越来越便利。实践中，我们通常可以用比上一节更简洁的代码来实现同样的模型。在本节中，我们将介绍如何使用tensorflow2.0推荐的keras接口更方便地实现线性回归的训练。\n",
    "\n",
    "## 生成数据集\n",
    "\n",
    "我们生成与上一节中相同的数据集。其中`features`是训练数据特征，`labels`是标签。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "pycharm": {
     "is_executing": false,
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "num_inputs = 2\n",
    "num_examples = 1000\n",
    "true_w = [2, -3.4]\n",
    "true_b = 4.2\n",
    "features = tf.random.normal(shape=(num_examples, num_inputs), stddev=1)\n",
    "labels = true_w[0] * features[:, 0] + true_w[1] * features[:, 1] + true_b\n",
    "labels += tf.random.normal(labels.shape, stddev=0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "虽然tensorflow2.0对于线性回归可以直接拟合，不用再划分数据集，但我们仍学习一下读取数据的方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from tensorflow import data as tfdata\n",
    "\n",
    "batch_size = 10\n",
    "# 将训练数据的特征和标签组合\n",
    "dataset = tfdata.Dataset.from_tensor_slices((features, labels))\n",
    "# 随机读取小批量\n",
    "dataset = dataset.shuffle(buffer_size=num_examples)\n",
    "dataset = dataset.batch(batch_size)\n",
    "data_iter = iter(dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor(\n",
      "[[ 0.11906993  1.853482  ]\n",
      " [ 1.4858664  -0.18852489]\n",
      " [-1.0006745  -0.40935215]\n",
      " [-0.25300497  0.60063976]\n",
      " [-0.28377545 -1.5488006 ]\n",
      " [ 0.27551156 -0.61813927]\n",
      " [-0.80954224  0.77936345]\n",
      " [-0.06401849 -0.0905276 ]\n",
      " [ 0.17151324  1.3873678 ]\n",
      " [ 1.2723187   0.66405845]], shape=(10, 2), dtype=float32) tf.Tensor(\n",
      "[-1.8673204   7.8212767   3.5861506   1.657495    8.903213    6.8579035\n",
      " -0.06758562  4.3810105  -0.16158912  4.485695  ], shape=(10,), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "for X, y in data_iter:\n",
    "    print(X, y)\n",
    "    break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "定义模型,tensorflow 2.0推荐使用keras定义网络，故使用keras定义网络\n",
    "我们先定义一个模型变量`model`，它是一个`Sequential`实例。\n",
    "在keras中，`Sequential`实例可以看作是一个串联各个层的容器。\n",
    "在构造模型时，我们在该容器中依次添加层。\n",
    "当给定输入数据时，容器中的每一层将依次计算并将输出作为下一层的输入。\n",
    "重要的一点是，在keras中我们无须指定每一层输入的形状。\n",
    "因为为线性回归，输入层与输出层全连接，故定义一层"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "from tensorflow import initializers as init\n",
    "model = keras.Sequential()\n",
    "model.add(layers.Dense(1, kernel_initializer=init.RandomNormal(stddev=0.01)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "定义损失函数和优化器：损失函数为mse，优化器选择sgd随机梯度下降\n",
    "在keras中，定义完模型后，调用`compile()`方法可以配置模型的损失函数和优化方法。定义损失函数只需传入`loss`的参数，keras定义了各种损失函数，并直接使用它提供的平方损失`mse`作为模型的损失函数。同样，我们也无须实现小批量随机梯度下降，只需传入`optimizer`的参数，keras定义了各种优化算法，我们这里直接指定学习率为0.01的小批量随机梯度下降`tf.keras.optimizers.SGD(0.03)`为优化算法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from tensorflow import losses\n",
    "loss = losses.MeanSquaredError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras import optimizers\n",
    "trainer = optimizers.SGD(learning_rate=0.03)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "loss_history = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "在使用keras训练模型时，我们通过调用`model`实例的`fit`函数来迭代模型。`fit`函数只需传入你的输入x和输出y，还有epoch遍历数据的次数，每次更新梯度的大小batch_size, 这里定义epoch=3，batch_size=10。\n",
    "使用keras甚至完全不需要去划分数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 1, loss: 0.577524\n",
      "epoch 2, loss: 0.010238\n",
      "epoch 3, loss: 0.000279\n"
     ]
    }
   ],
   "source": [
    "num_epochs = 3\n",
    "for epoch in range(1, num_epochs + 1):\n",
    "    for (batch, (X, y)) in enumerate(dataset):\n",
    "        with tf.GradientTape() as tape:\n",
    "            l = loss(model(X, training=True), y)\n",
    "        \n",
    "        loss_history.append(l.numpy().mean())\n",
    "        grads = tape.gradient(l, model.trainable_variables)\n",
    "        trainer.apply_gradients(zip(grads, model.trainable_variables))\n",
    "    \n",
    "    l = loss(model(features), labels)\n",
    "    print('epoch %d, loss: %f' % (epoch, l))\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "下面我们分别比较学到的模型参数和真实的模型参数。我们可以通过model的`get_weights()`来获得其权重（`weight`）和偏差（`bias`）。学到的参数和真实的参数很接近。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([2, -3.4], array([[ 1.9945697],\n",
       "        [-3.3903365]], dtype=float32))"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "true_w, model.get_weights()[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4.2, array([4.1920047], dtype=float32))"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "true_b, model.get_weights()[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[39.801727,\n",
       " 53.19048,\n",
       " 14.46788,\n",
       " 39.947865,\n",
       " 24.486965,\n",
       " 30.713726,\n",
       " 39.17563,\n",
       " 13.047705,\n",
       " 33.28475,\n",
       " 20.761524,\n",
       " 18.420874,\n",
       " 34.008083,\n",
       " 14.405246,\n",
       " 17.041443,\n",
       " 7.005189,\n",
       " 18.382719,\n",
       " 18.695303,\n",
       " 13.632373,\n",
       " 12.635169,\n",
       " 20.074924,\n",
       " 15.94778,\n",
       " 9.27402,\n",
       " 10.427583,\n",
       " 8.54941,\n",
       " 16.81535,\n",
       " 6.8781943,\n",
       " 15.035431,\n",
       " 11.794742,\n",
       " 11.860071,\n",
       " 3.8811011,\n",
       " 7.9945374,\n",
       " 7.8730416,\n",
       " 9.143669,\n",
       " 10.242535,\n",
       " 5.7133584,\n",
       " 12.0080595,\n",
       " 5.0345087,\n",
       " 5.8301377,\n",
       " 6.3047495,\n",
       " 7.8470206,\n",
       " 10.113134,\n",
       " 9.405206,\n",
       " 8.132036,\n",
       " 5.154071,\n",
       " 4.421684,\n",
       " 5.987176,\n",
       " 5.059777,\n",
       " 5.080542,\n",
       " 3.4634635,\n",
       " 2.47151,\n",
       " 5.6223817,\n",
       " 4.015649,\n",
       " 3.8196254,\n",
       " 3.3439813,\n",
       " 3.7710037,\n",
       " 3.2606869,\n",
       " 0.9403268,\n",
       " 1.5239546,\n",
       " 2.5819716,\n",
       " 3.9055488,\n",
       " 6.8448067,\n",
       " 2.0272841,\n",
       " 3.6449275,\n",
       " 4.1734414,\n",
       " 3.8974013,\n",
       " 1.9378322,\n",
       " 3.0132825,\n",
       " 2.4585395,\n",
       " 1.1937939,\n",
       " 1.9572703,\n",
       " 1.2297701,\n",
       " 0.69520515,\n",
       " 1.5201247,\n",
       " 1.3764166,\n",
       " 1.4739829,\n",
       " 1.7456497,\n",
       " 0.68866235,\n",
       " 2.0207152,\n",
       " 1.0318539,\n",
       " 1.1638341,\n",
       " 1.3316251,\n",
       " 0.6911205,\n",
       " 1.9045906,\n",
       " 1.5170772,\n",
       " 0.81408226,\n",
       " 1.0188065,\n",
       " 0.75774074,\n",
       " 1.2050498,\n",
       " 0.72318894,\n",
       " 1.0053222,\n",
       " 0.5887068,\n",
       " 1.4932233,\n",
       " 1.2776058,\n",
       " 0.36616892,\n",
       " 0.9163977,\n",
       " 0.98734426,\n",
       " 0.830197,\n",
       " 0.37068218,\n",
       " 0.6642739,\n",
       " 0.55620503,\n",
       " 0.39622185,\n",
       " 0.20022216,\n",
       " 0.79677105,\n",
       " 0.29217798,\n",
       " 0.57444954,\n",
       " 0.73432416,\n",
       " 0.51740295,\n",
       " 0.5292425,\n",
       " 0.31934088,\n",
       " 0.5081655,\n",
       " 0.15677172,\n",
       " 0.349867,\n",
       " 0.35969028,\n",
       " 0.5003898,\n",
       " 0.4415594,\n",
       " 0.1803854,\n",
       " 0.3501873,\n",
       " 0.33100772,\n",
       " 0.15929379,\n",
       " 0.26835275,\n",
       " 0.17359123,\n",
       " 0.13071862,\n",
       " 0.13897128,\n",
       " 0.23311763,\n",
       " 0.17250268,\n",
       " 0.28053337,\n",
       " 0.19408342,\n",
       " 0.16487274,\n",
       " 0.16594473,\n",
       " 0.21566626,\n",
       " 0.2038723,\n",
       " 0.18577202,\n",
       " 0.36484432,\n",
       " 0.14714375,\n",
       " 0.12133932,\n",
       " 0.08995683,\n",
       " 0.11307355,\n",
       " 0.2692285,\n",
       " 0.09830435,\n",
       " 0.09511658,\n",
       " 0.16965504,\n",
       " 0.1323972,\n",
       " 0.07526772,\n",
       " 0.12022765,\n",
       " 0.13092557,\n",
       " 0.11130451,\n",
       " 0.16388243,\n",
       " 0.1287101,\n",
       " 0.11105399,\n",
       " 0.04262692,\n",
       " 0.08649181,\n",
       " 0.03936447,\n",
       " 0.12013745,\n",
       " 0.033055924,\n",
       " 0.05495699,\n",
       " 0.05561447,\n",
       " 0.030995494,\n",
       " 0.06407594,\n",
       " 0.059013575,\n",
       " 0.09031354,\n",
       " 0.057709478,\n",
       " 0.039839227,\n",
       " 0.037912793,\n",
       " 0.058507122,\n",
       " 0.064165235,\n",
       " 0.039651264,\n",
       " 0.06532028,\n",
       " 0.056082767,\n",
       " 0.034878023,\n",
       " 0.04585194,\n",
       " 0.015101899,\n",
       " 0.019507688,\n",
       " 0.03404371,\n",
       " 0.013743378,\n",
       " 0.022505464,\n",
       " 0.027803179,\n",
       " 0.018081311,\n",
       " 0.008088188,\n",
       " 0.020442586,\n",
       " 0.048676144,\n",
       " 0.021925796,\n",
       " 0.009916232,\n",
       " 0.014368388,\n",
       " 0.018147377,\n",
       " 0.018494088,\n",
       " 0.01463414,\n",
       " 0.023203274,\n",
       " 0.01918884,\n",
       " 0.02973583,\n",
       " 0.016088527,\n",
       " 0.025977403,\n",
       " 0.023255372,\n",
       " 0.011429547,\n",
       " 0.019745022,\n",
       " 0.017115321,\n",
       " 0.006535507,\n",
       " 0.010758871,\n",
       " 0.006769347,\n",
       " 0.006235889,\n",
       " 0.020224262,\n",
       " 0.005100709,\n",
       " 0.00983331,\n",
       " 0.013380116,\n",
       " 0.021389257,\n",
       " 0.012816164,\n",
       " 0.00917027,\n",
       " 0.0038917877,\n",
       " 0.0039019454,\n",
       " 0.0042894883,\n",
       " 0.0042790775,\n",
       " 0.00939782,\n",
       " 0.006586154,\n",
       " 0.008569246,\n",
       " 0.003474607,\n",
       " 0.005878182,\n",
       " 0.0056525464,\n",
       " 0.0034994557,\n",
       " 0.0061428146,\n",
       " 0.0006752965,\n",
       " 0.004038655,\n",
       " 0.009399179,\n",
       " 0.0024249922,\n",
       " 0.0053922795,\n",
       " 0.0045819324,\n",
       " 0.002530494,\n",
       " 0.0038768717,\n",
       " 0.004603903,\n",
       " 0.0015568486,\n",
       " 0.00058786024,\n",
       " 0.0041333837,\n",
       " 0.0068843067,\n",
       " 0.002691524,\n",
       " 0.0037159987,\n",
       " 0.0012608811,\n",
       " 0.00663739,\n",
       " 0.0020495593,\n",
       " 0.002227059,\n",
       " 0.003164728,\n",
       " 0.0016560906,\n",
       " 0.0010445726,\n",
       " 0.003003891,\n",
       " 0.0028429998,\n",
       " 0.0012617535,\n",
       " 0.0020194445,\n",
       " 0.002766458,\n",
       " 0.0020570352,\n",
       " 0.0018062912,\n",
       " 0.001121258,\n",
       " 0.0005798013,\n",
       " 0.0015520301,\n",
       " 0.00050824275,\n",
       " 0.0023863157,\n",
       " 0.00095969776,\n",
       " 0.0009161158,\n",
       " 0.0015514639,\n",
       " 0.0011714095,\n",
       " 0.00049272657,\n",
       " 0.0011581723,\n",
       " 0.001619019,\n",
       " 0.00023566847,\n",
       " 0.00038945713,\n",
       " 0.0006287855,\n",
       " 0.0012527744,\n",
       " 0.00088742917,\n",
       " 0.0007648802,\n",
       " 0.00059341005,\n",
       " 0.0012123943,\n",
       " 0.0007128694,\n",
       " 0.0010048638,\n",
       " 0.0005971165,\n",
       " 0.0009232084,\n",
       " 0.0005925436,\n",
       " 0.000410921,\n",
       " 0.000574462,\n",
       " 0.00061742106,\n",
       " 0.00033323184,\n",
       " 0.0009302815,\n",
       " 0.0007288108,\n",
       " 0.00038465165,\n",
       " 0.00022217425,\n",
       " 0.00048612308,\n",
       " 0.00054423406,\n",
       " 0.00035819947,\n",
       " 0.0005609441,\n",
       " 0.0005526895,\n",
       " 0.00028165616,\n",
       " 0.00047497722,\n",
       " 0.0004148931,\n",
       " 0.00020819406,\n",
       " 0.00059870596,\n",
       " 0.00028783423,\n",
       " 0.0003370984,\n",
       " 0.00018644368,\n",
       " 0.000553568,\n",
       " 0.00022104204,\n",
       " 0.0005205969,\n",
       " 0.0005518681,\n",
       " 0.00017890071,\n",
       " 0.00013345432,\n",
       " 0.00030553862]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loss_history"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "metadata": {
     "collapsed": false
    },
    "source": []
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
